Computer Vesion Based Music Information Retrieval

This project relies on computer vision techniques to build a practical music retrieval system. Our approach tackles traditional music identification as a corrupted sub-image retrieval problem from the 2-D spectrogram representation of the original songs. More specifically, a query snippet spectrogram is matched against our database using its descriptor representation. We utilize a novel pairwise boosting method to learn a robust and compact subset of Viola-Jones filters. This subset of filters captures distinctive information of each spectrogram while being resistant to distortion. By applying these filters, we can transform a spectrogram into a low-dimension binary representation. During the query phase, we search for all the candidates that locally match the descriptors of the query snippet. Then, we select the most likely candidate using an occlusion model based on a 2-state HMM. In order to test the performance of our music retrieval system, we rely on a case study: the Reggae music genre of the iTunes Store. The recognition is both fast and accurate, even with noisy background and a short song snipped as query. We believe that this project can have a significant impact on the way people organize and research music. Indeed, this system tags songs automatically, adding music meta data such as album, artist, genre, and art cover. Furthermore, this project also opens up new avenues for live music search and identification.